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| J NTRODUCTION 


In the last decade Ontario educators have 
been paying considerable attention to the 
related areas of accountability and con- 
trolled change in the educational system. 
As a culmination of the experience gained 
through ten years of study, the ministry 
has developed the Co-operative Evalua- 
tion and Development of School Systems 
(CEDSS) model. The implementation of 
CEDSS in a school system will assist 
trustees, educators, and the community in 
addressing the accountability issue and will 
provide information on the strengths and 
weaknesses in the educational system. In 
addition, it will provide a basis for effective 
decision-making and the development of 
appropriate follow-up activities. This man- 
ual has been prepared for the use of all 
personnel who will be involved in the eval- 
uation and development of a school system. 


The CEDSS model has three primary goals: — 

— to provide reliable information so 
that achievements may be more clearly 
perceived; 

— to collect information that will provide 
the basis for effective decision-making; 
and : 

— to provide guidance for system planning 
so that existing strengths can be reinforced 
and activities that will lead to positive 
change developed. 


Underlying the CEDSS model are the 


following seven basic principles, which are 

recommended to all users: 

— Involvement. Anyone who may be affected 
by the evaluation and development pro- 
cesses should be integrally involved in 
them. 

—- Communication. In order that maximum 
benefits may be derived from the evalua- 
tion and development processes, effective 
communication should be maintained 
among all participants. 

- Self-evaluation. Sound evaluation prac- 
tices should include opportunities for self- 
evaluation. 

— Co-operation. Since all members of the 
community, at whatever levels of author- 
ity and function, are concerned with the 
quality of education, all levels should be 
involved in the evaluation and develop- 
ment processes. 

— Comprehensiveness. An evaluation and 
development model must recognize all of 
the related components that operate 
within the school system. 

- Continuity. Evaluation and development 
processes must be continuous so that infor- 
mation, assessment, and decisions can 
emerge at any stage. 

— Co-ordination. All components of the 
evaluation and development model must 
be valid, well planned, and systematically 
implemented. 
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| BRIEF OUTLINE OF 
T 


HE CEDSS MODEL 


The CEDSS model includes two major 
activities — evaluation and development. 
The basic components of the model, which 


_are described in greater detail in the proce- 


- dural section of this manual, are outlined 


in figure 1. 


Figure 1: Model for the Co-operative Evaluation 
and Development of School Systems 
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. Initiation. The school board submits a 


resolution to the ministry expressing its 
wish to proceed with a co-operative 
evaluation. The nature of the project is 
then publicized and discussed in the 
schools and the community with the 
primary aim of eliciting the support and 
participation of individuals. This is 
followed by the ministry’s appointment 
of a project co-ordinator after consulta- 
tion with the chief executive officer of 
the board. The board then appoints an 
internal evaluation chairperson and 
designates a senior education official to 
assume overall local responsibility for the 
implementation of the CEDSS model. 


. Internal evaluation. An evaluation of 


selected areas of the educational system 
is conducted with the involvement of 
staff, trustees, students, parents, and the 
community. 


. External evaluation. An external evalua- 


tion of the same selected areas of the 
school system is conducted by an exter- 
nal evaluation team. This evaluation 
provides another view of the school 
system. 


4. Co-operative review meeting. The chair- 


persons of the internal committee and the. 


external team and their respective groups 
meet with the project co-ordinator to 
discuss areas of agreement and differences 
in evaluation findings. _ 


5. Final evaluation report. Using the internal 


and the external reports and the results 
of the co-operative review, the project 


co-ordinator prepares a final report. This _ 


is then presented to the board. . 
6. Analysis and response. With the guidance 
of the senior board official, an A 
and response committee, representing | 
all members of the school system, is 
charged with the task of analysing the 


final report and making recommendations 


for follow-up action. | 

7. Implementation. Follow-up plans are im- 
plemented under the leadership of the 
senior board official and with the support 
of a monitoring committee representing 
school-system personnel. These follow-up 
plans may include the implementation 
of new programs or the modification of 
existing programs. 

8. Incorporation. On the basis of the infor- 
mation received by the monitoring com- 
mittee, the senior board official makes 
recommendations to incorporate, cancel, 
or further revise the new or adapted 
programs. 


Although the CEDSS model has clearly 
delineated stages, it is flexible and practical. 
It provides a sound basis for planning change 
and for identifying and conserving strength. 
It permits the shaping, within each stage, 
of organizational activities that are highly 
relevant to the particular characteristics of 
the system undertaking the evaluation 
project. 


| This process was known initially as the 


_ of two research reports and of feedback 


A BRIEF HISTORY OF | 
“ “THE CEDSS MODEL ” 


For a decade the Ministry of Education ha | 
co-operated with boards of education on ani 
optional basis in system-evaluation projects. © 


Co-operative Evaluation of School Systems © 
(CESS). The refined process is now known 
as the Co-operative Evaluation and Develop- ‘ 
ment of School Systems (CEDSS). i 


The original evaluation model involveda _ 
six-step process. On the basis of the results ~ 


from various participants in the pilot proj- 
ects, the model was revised to an eight- 
step process. The first five stages of the 
original evaluation model were retained and — 
were complemented by a three-stage devel- — 
opment component designed to facilitate 
follow-up activities. 


research study of the eleven pilot projects 
undertaken from 1972 to 1976 was con- 
ducted by an outside investigator and 
teleased under the title Improving Ontario. 
Education: An Analysis of the Co-operative 
. Evaluation of School Systems (1972-1976) .} 
A second study of the implementation of 
i the co-operative evaluation system in six 
additional boards from 1976 to 1978 speci- 
fically addressed modifications made to the 
original evaluation model — variations in 
implementation strategies and in the report- 
‘ing format. This research was published by the 
ministry under the title School Systems Co- 
operative Evaluation Analysis: 1976-1978.2 
. urrent users of the model may find the 
descriptions of these earlier projects useful 
. to the design and implementation of their 
“own evaluation projects. 


i. Doris Ryan, Improving Ontario Education: An Analy- 

is of the Co-operative Evaluation of School Systems 

(1972 -1976) (Toronto: eridy of Education, Ontario, 

1 1976). 

2. Doris W. Ryan, School Systems Co-operative ae 

tio Analysis: 1976-78, Review and Evaluation Bulle- 
il SEE. (earns Ministry of Education, 


| “HE OPERATIONAL 


STAGES OF THE 
CEDSS MODEL ~ 


EVALUATION 
COMPONENT 


The evaluation phase of the CEDSS model 
begins with the activities of the initiation 
stage and is completed with the presentation 
of the final evaluation report to the board. 
A possible time line for the completion of 
each stage of the evaluation phase is pre- 
sented in Appendix D. 


1. INITIATION 


The initiation stage includes (a) the pre- 
liminary information sessions arranged by 
the ministry at the board’s request, 

(b) system-awareness activities, (c) the 
board’s resolution to participate in a 
co-operative evaluation and development 
project, (d) the ministry’s response, and 
(e) the key leadership appointments. 


a) Preliminary information sessions. The 

school board begins the initiation stage by © 
requesting information from the ministry on 
the CEDSS model. A ministry representative 


meets with the board or its contact group to - 


describe the nature of the model and to 
respond to questions regarding its implemen- 
tation. In this exploratory meeting a number 
of communication strategies can be created 
to develop a broad base of awareness within 
the system. 


b) Activities to promote awareness within the 
system. Experience to date indicates that 
this is a crucial phase of the initiation stage. 
The ultimate success of the co-operative 
evaluation and development project depends 
largely on the willingness of the individuals 
within the system to participate. Adequate 
time and preparation for communicating 
the nature of the project should, therefore, 
be a primary concern. It is important that 
this communication emphasize that the 
evaluation is to include only selected areas 
within the system as a whole. Both the 
internal and external evaluations are con- 
cerned with making objective, concurrent 
assessments of the effectiveness of selected 
aspects of the system and not with the 
evaluation of individuals within the system. 


The following are some of the many strate-_ 
gies that may be used to increase awareness 
of the evaluation within the system and 
the community: 


~ An initiation team (see Appendix B) can 

- be established to study the model and 
discuss it with school staffs, parent and 
community organizations, administrators, © 
and student groups. ’ 

— An evaluation forum, at which he model 
is presented and discussed with those 
present, can be conducted for each family — 
of schools in the system. 

— The project can be explored as part of a 
professional activity day. 

— A question-and-answer videotaped pro- 
gram can be circulated to all schools and 
to interested groups. 


In some projects the major thrust of the 
strategies designed to effect awareness within — 
the system and the community may take 
place after the board has approved the eval- — 
uation project. Regardless of the timing, 
the importance of laying a common ground- 
work of perceptions and expectations and of © 
developing a full understanding of the model ; 
cannot be overstressed. 


c) Board resolution. When the board has 
made its decision to proceed, it passes a ten- 
tative resolution to this effect and forwards 
a copy to the ministry’s regional director 
of education. 


d) Ministry approval. The ministry reviews 
the communication activities of the board 
and the availability of ministry resources, 

makes a decision to accept or postpone the 
application, and notifies the board. After 
receiving ministry approval, the board passes © 
a final resolution of commitment to the 
implementation of the co-operative evalua- — 
tion and development project. 


e) Leadership appointments. Several impor- 


tant leadership appointments are now made. 


A project co-ordinator is appointed by the 
ministry to assume responsibility on its 
behalf for the co-operative evaluation. 

An assistant project co-ordinator may also 
_ be appointed. Before they are finalized, 

‘. these appointments are discussed with the 
chief executive officer of the board. 


The project co-ordinator works closely 
with the internal evaluation chairperson 
throughout the project. He/she suggests 
possible activities and strategies; provides a 
link to the resources of the ministry; offers 
existing evaluation instruments; assists in 
the development of new evaluation instru- 
ments; and so on. A role description and 
suggested procedural schedule for the pro- 
ject co-ordinator and the assistant project 
co-ordinator are provided in Appendix B. 


ee RE a 


le 
»: 


LJ 
= 


_ Figure 2: A Possible Organizational Structure 
_ for a Board’s Implementation of the Evaluation 
Phase of CEDSS 
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The board makes two appointments that 
are crucial to the success of the project. 

A senior education official of the board is 
given local responsibility for the implemen- 
tation of the CEDSS model. A chairperson 
for the internal evaluation committee is 
also formally appointed by the board to lead 
and assume responsibility for the internal 
evaluation. Selections for this position 

are usually made from among middle- 
management personnel through open com- 
petition, with the appointee seconded from 
regular duties for the intensive activity 
period of the project. A suggested procedural 
schedule for the internal evaluation chair- 
person is included in the role descriptions 
section of the manual (Appendix B). In 
school systems in which French is the 
language of instruction in some schools, 
consideration should be given to the 
appointment of co-chairpersons, one ~ 
English-speaking and one French-speaking. 


A possible organizational structure for the 
implementation by a school system of the 
evaluation phase of CEDSS is shown in 


figure 2. A detailed outline of the role and 


function of each individual or group depicted 
is provided in Appendix B. 
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2. INTERNAL Lihicmsed tanec 


The eat Te staue ae: the 
school system with an opportunity for self- 
evaluation. It includes (a) an expansion of 
the activities of the initiation stage to effect 
awareness of the evaluation within the 


system, (b) a determination of the areas to _ 


be evaluated, (c) the selection and orienta- 
tion of the internal evaluation committee 
and subcommittees, (d) the gathering of 
data, and (e) the preparation of the inter- 
nal evaluation report. 


a) Effecting awareness of the evaluation within — 
the system. The task of increasing awareness. 


within the system and the community is 
usually assigned to the internal evaluation 
chairperson. The procedures employed for — 
this purpose in the initiation stage may be 
expanded and intensified not only as a 
means of effecting greater awareness but _ 
also with a view to obtaining reactions to 
the project and input regarding possible. 
areas to be evaluated. | 


b) Determining areas to be evaluated. Before 
the evaluation process itself can proceed, 
areas to be evaluated across the total system 
must be selected. The successful implemen- 
tation of the procedures suggested for 
effecting awareness within the system and 
the community will have already yielded 
reactions to the project and some input 
regarding areas to be evaluated. The follow- 
ing are some specific strategies for the devel- 
opment of an initial list of possible areas to 
be evaluated: | 


— the polling of all levels of school system 
personnel and as many members of the 
community as time and resources permit; 

— the solicitation of suggestions from the 
chief executive officer of the board, the 
chairperson of the board, and each of the 
formal organizations within the system; 

~ the invitation to parents and community 
members (through such means as letters, 
news bulletins, and newspaper advertise- 
ments) to list areas that they would like 
to see evaluated. | 


_ Each of these strategies will heighten 
awareness of and interest in the CEDSS 


- possible areas for evaluation. For the final 


= The list of areas to be evaluated should be — 


~- A new list should be formulated, based 


project as well as providing an initial lis 


determination of areas to be evaluated the 
following steps are suggested: ! 


~ An initial list of possible areas to be eval- — 
uated should be circulated. Opportunities — 
should be provided for forum discussions | 
of the project, and the submission of addi- 
tional suggestions should be encouraged. © 


revised to include any additional sugges- 
tions made by school system personnel, 

parents, or community members. This 
revised list should then be circulated 
throughout the system so that individuals 
can rate the areas according to their 
perceived level of importance for inclusion 
in the CEDSS project. 


on the survey, of a specific number of the 
top-ranked areas to be evaluated. The 
final choice of areas to be evaluated is 
made from this list by the internal 
evaluation committee. 


Once the list of areas to be evaluated has 
been established, it is communicated to 
personnel throughout the system and to 


_ parents and community members as a part 


of the continuing effort to effect awareness 
within the system and the community. 


c) Selecting the internal evaluation committee 
and subcommittees. The internal evaluation 
is designed and conducted by the internal 
evaluation committee and the project co- 
ordinator. The following are some of the — 
patterns into which the internal evaluation 
committee and its subcommittees can be 
organized: | 


- Single representatives from each school, 
the board administration, parent groups, 
and the support:staff form the executive 
council of the internal evaluation commit- 
tee. This executive council then estab- 
lishes a series of subcommittees, which 
are concerned with the various areas to 
be evaluated (attitudes, roles, subject dis- 
ciplines, and so on). 

- The internal evaluation committee is made 
up of designated representatives of the 
formal organizations within the system — 
the various federations of teachers, super- 
visory officers, support-staff unions, 
trustees’ associations, parent associations, 
and student councils — and the deter- 
mination of subcommittees is then under- 
taken by the committee members. 

- The internal evaluation committee chair- 
person — through notices or visits to all 
schools, parent groups, community groups, 
and school or system associations — calls 
for volunteers for the internal evaluation 
committee. The committee and its sub- 
committees are then constructed from the 
volunteers who emerge. 


The internal evaluation committee is com- 
posed of the internal evaluation chairperson, 
the senior board official, the project co- 
ordinator, and a representative from each 
subcommittee. Each subcommittee is as- 
signed one area for evaluation and includes 
one or more individuals directly involved in 
the area undergoing evaluation. Additional 
subcommittees with relevant interests and 
expertise are appointed as needed. 


In large school systems that have a research 
director and research staff it is suggested 
that the research director play a major role 
in the evaluation project. Appendix B 
includes a description of a possible role for 
the research director. 


A general orientation program to the CEDSS 
project is conducted by the internal chair- 
person in co-operation with the project co- 
ordinator. Workshops in specific evaluation 
techniques such as questionnaire develop- 
ment are provided as needed. 


Each subcommittee is usually responsible 
for conducting its own data analysis. Data 
processing of questionnaires may be arranged 
by the project co-ordinator. Requests for 
this assistance are channelled through the 
internal chairperson. The final evaluation 
plans and data-collection procedures are 
approved by the internal evaluation 
committee. 


d) Gathering data. A variety of data- 
gathering techniques are reviewed by the 
internal evaluation committee. 


Modification of existing survey instruments 
or the creation of entirely new instruments 
and questionnaires may be necessary to fit 
the particular needs or characteristics of 
the system being evaluated. Assistance in 
the local development or refinement of in- 
struments is available from the project co- » 
ordinator in the form of in-service training 
in questionnaire design for members of the 
internal evaluation committee. The project 
co-ordinator is also able to provide access to 
the survey items used in other evaluation 
projects. Alternatively, the board may 
negotiate with the Ontario Institute for 
Studies in Education or with faculties of 
education for the development of more com- 
plex questionnaires and for other types of 
assistance. 


The evaluation instruments available from 
the ministry are in four sets — system-review 
surveys, role-review surveys, program-review 
surveys, and student-achievement instru- 
ments. 


i) System-review surveys are designed for 
wide distribution within the system. 
The instruments are used by sample 
populations representing teachers, 
students, and parents. The results pro- 
vide descriptive indicators as to the 
state of the educational system at large, 
how the system feels about itself as an 
educational enterprise, and about edu- 
cation in general. 


ii) Role-review surveys are available for four 
roles in the system — those of trustee, 
teacher, principal, and administrator. 
The survey form has been designed to 
gather some general information about 
the role being surveyed. Its use may be 
supplemented by personal interviews 
with a selected sample of respondents. 
Confidentiality is respected, and re- 
spondents are not required to identify 
themselves. A handbook for the use of 
the instruments, including alternative 
strategies for data gathering and sugges- 
tions about data interpretation, is avail- 
able through the project co-ordinator. 
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iii) Program-review surveys are designed for 
use in both elementary and secondary 
schools and can be used at both the 
individual school and the system level. 
The original format provides a general 
overview of the program for any speci- 
fied subject, and modifications are 
possible to make the surveys more 
specific. 


The handbook User’s Guide to the Program 
Review Surveys for Elementary and Secondary 
Schools? outlines the steps to be followed 
when the survey forms are employed and 
offers suggestions for adaptation. Another 
section of the handbook describes a frame- 
work for writing a report using data gathered 
from the review process. 


Although questionnaires are a quick method 
of collecting data, they cannot gather 
detailed information to the extent that can 
be done by interviewing. The initial infor- 
mation assembled through the use of the 
three sets of review guides can be followed 
up with group discussions, interviews, on- 
site observations, and meetings to confirm 
or refine the data findings. Immediate feed- 
back to personnel in the system has the 
potential to initiate a change in activities at 
the outset of the evaluation exercise. 


iv) Student-achievement data. Until recently 
the CEDSS model has not included in 
its design any activity to secure measures 
of student achievement. In many juris- 
dictions a system analysis of existing 
data can be undertaken. The internal 
evaluation committee is encouraged to 
consider the inclusion of instruments 
from the Ontario Assessment Instru- 
ment Pool. 


3. John E. Davis, User’s Guide to the Program Review 
Surveys for Elementary and Secondary Schools (Toronto: 
Ministry of Education, Ontario, 1980). 


Instruments developed for the Ontario 
Assessment Instrument Pool reflect the 
goals, aims, and objectives of selected 
ministry curriculum guidelines. The wide 
variety of assessment methods and in- 
struments contained in the pool serves 
two basic functions: 


— to assist program evaluation at the 
provincial and local levels; and 

— to assist in the evaluation of student | 
achievement at the classroom level 
for both formative and summative 
evaluation. 


‘Instrumentation from the pool is availa- 
ble for use as a component of the inter- 
nal evaluation. | 


e) Preparing the internal evaluation report. 
The internal evaluation report is prepared 

_ by the chairperson of the internal evalua- 
tion committee and is reviewed by the com- 


- mittee members. Each subcommittee is 


usually responsible for submitting to the 
internal evaluation chairperson a report 
of its evaluation. This report includes a 
statement of objectives, a brief account of 
the procedures used for the collection and 
analysis of data, the results of the evalua- 
tion, and any recommendations for system 
changes. These detailed reports assist the 
internal evaluation chairperson in the 
preparation of the internal report. The report 
summarizes the subcommittee findings and 
_ recommendations, and an attempt is made 
_ to place the recommendations in order of 
_ priority. 


It is usual to include in the report an 
introductory section that describes the 
development of the system, the geographic 
area served by the board, and the social, 
cultural, and economic characteristics of 
the region. Background information on the 
review process, the committee organization, 
and the procedures used for the collection 
and analysis of data are also included. The 
last section of John Davis’s User’s Guide to 
_ the Program Review Surveys for Elementary 
and Secondary Schools outlines a framework 
_ for writing a report using data gathered 
from the review process. 


To give a more complete picture of the 
system, the senior board official may wish 
to provide a section in the internal report 
describing the achievements of the system 
in the previous five years. 


The internal evaluation report is filed with 
the project co-ordinator in sufficient quantity 
for use in orienting the external evaluation 
team to the system (except where it has 
been agreed that the external team is to 
operate in a goal-free fashion, as described 
under “External Evaluation” below). 


f) Funding for CEDSS projects. Where a board 
negotiates an agreement with the Minister 
for a comprehensive co-operative school 
system review, the board is paid for the 
expenditures involved in the preparation of 
the internal, external, and final reports to a 
maximum amount as defined in the Regu- 
lations, General Legislative Grants. 


The completed form “Grant for a Co- 
operative Evaluation and Development of 
School Systems Project-Internal Evaluation 
Report” is to be submitted after a board has 
completed the internal evaluation stage of 
the co-operative review. The form will be 
signed by the senior board official and sub- 
mitted for approval to the regional director 
of education. The regional director will send 
a letter indicating approval to the board and 
forward copies of the letter and the com- 
pleted forms to the Grants Services Section 
of the Financial Services Branch of the 
ministry. The board will then receive 
reimbursement of approved expenses up to 
the maximum defined by the Regulations, 
General Legislative Grants. 
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The completed form “Grant for a Co- 
operative Evaluation and Development of 
School Systems Project—External Report 


_and Final Report” is to be completed after a 


AZ 


board has completed the external evaluation 
stage of the co-operative review. The form 
will be signed by the senior board official 
and submitted for approval to the regional 
director of education. The regional director 
will send a letter indicating approval to the 
board and forward copies of the letter and 
the completed forms to the Grants Services 
Section of the Financial Services Branch of 
the ministry. The board will then receive 
reimbursement of approved expenses up to 
the maximum defined by the Regulations, 
General Legislative Grants. 


3. EXTERNAL EVALUATION. 


The essential purpose of the external evalua- 
tion is to provide another view of the school 
system. In addition, the conclusions of the 
external team may provide possible solutions 
to problems raised but left unanswered by 
the internal report or may point out signifi- 
cant new problem areas. 


Towards the end of the internal evalua- 
tion phase the project co-ordinator recruits 
an external evaluation chairperson and an 
evaluation team after consultation with the 
internal chairperson and the senior board 
official. The external team, which consists 
of persons with expertise relevant to the 
concerns expressed by the internal team 
and the areas to be evaluated, may include 
ministry staff, personnel from the Ontario 
Institute for Studies in Education, members 
of faculties of education, personnel from 
other school boards, and individuals exter- 
nal to the school system (e.g., from the 
business community) as appropriate. The 
role of the external team is described in 


Appendix B. 


| of activities prior to an on-site visitation. 


‘meetings, parent interviews, classroom visits, — 


person the external team undertakes a series | 
These include a study of the report of the 


be evaluated, a session devoted to planning — 
the schedule for the visit, and a discussion 
to orient the team to the system. Plans for 
visitation activities, which include staff 


board meetings, and so on, are reviewed, 
and a format of on-site activities is estab- 
lished. The external evaluation chairperson — 
consults with the internal chairperson and 
the senior board official to determine the 
means to prepare the system for the team 
Visit. 


The external evaluation team spends a 
maximum of ten days in the school system. — 
At least one day is devoted to the writing of © 
the external report. The report is written 
from a discrepancy-analysis perspective, in 
that it indicates discrepancies between the 
observations of the external team andthe 
information in the internal report, although ~ 
new discussion areas may also be introduced. ~ 
The format is usually similar to that of the 
internal report to facilitate comparison and 
later references. The completed external 
evaluation report is submitted to the project © 
co-ordinator. 


A variation is possible in the external 
evaluation stage. The internal evaluation 
report can be withheld from the external 
evaluation team. This strategy is agreed to 
at the beginning of the exercise. Although 
the information gathered in this way can 
give extra emphasis to many aspects of the 
internal report, it does raise the possibility 7 
that evaluation gaps might occur within the — 
total evaluation. 


_ 4. CO-OPERATIVE REVIEW MEETING 


The purpose of the co-operative review 
meeting is to give members of both evalua- 
tion teams an opportunity to review their 
respective findings, interpretations, and 


_ recommendations. As part of the review 


process, previous conclusions may be modi- 
fied, and different suggestions may be made. 


The co-operative review meeting is called by 
the project co-ordinator after the internal 
evaluation committee and the external eval- 


- uation team have each had an opportunity 


to review both the internal and the external 


_ reports. Ideally, all of the members of both 
_ the internal and the external teams partici- 
_ pate actively in this final dialogue, in which 


all of the evaluation data are discussed. 


_ The meeting should ideally take two days 


and should be chaired by the project co- 


ordinator. A number of approaches to the 
_ agenda and the conduct of the co-operative 


review meeting are possible. The ultimate 
goal of the meeting is to clarify in detail the 
areas of agreement and disagreement be- 


- tween the internal and external evaluation 


\ 


eae 


/ 


reports. 


Two people, one representative from each 
of the internal and external evaluation 


- teams, are assigned the task of keeping 


minutes for the co-operative review meeting. 
These minutes are later submitted to the 


_ project co-ordinator for use in writing the 


final evaluation report. 


5. FINAL EVALUATION REPORT 


The compilation of the final report is the 


_ responsibility of the project co-ordinator. 


It is a delineation of the strengths and weak- 
nesses of the system. Recommendations for 
changes are highlighted for consideration 


by the board. 


The final evaluation report usually includes 
the following: 


a description of the areas of agreement 

and disagreement between the internal 

and external evaluation teams; 

recommendations from the project 

co-ordinator; 

— suggestions for follow-up activities; 

— conclusions; 

— asummary of recommendations from all 
sources; | 

~- the complete reports of the internal and 
external evaluation teams; and 

— areport of the co-operative review 

meeting. 


The final evaluation report is presented 

to the senior board official and the chief 
executive officer of the board in advance of 
its presentation to the board. Because of 
the responsibilities of the senior board 
official in the development activities, it is 
advisable for the project co-ordinator to 
spend considerable time discussing with the 
senior board official the significant issues 
contained in the report and the recommen- 
dations for follow-up activities. 


The presentation of the final evaluation 
report to the board is made by the project 
co-ordinator in co-operation with the senior 
board official and with the support of the in- 


ternal and external evaluation chairpersons. 


The actual presentation may take place 
over several sessions (formal presentation, 
committee study, full board review, etc.). 


Six copies of the final report are to be for- 
warded through the project co-ordinator to 
the Ministry of Education. 
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DEVELOPMENT 
COMPONENT 


The development component is the second 
of the two distinct components of the 
CEDSS model. If the project’s continuity is 
to be ensured, the development phase should 
begin as soon as possible after the comple- 
tion of the evaluation phase. The system- 
awareness strategies of the initiation stage 
will have stimulated a high level of interest 
in the contents of the evaluation report. As 
well, the extensive involvement of system 
personnel in the evaluation phase will have 
generated a high level of commitment to 
follow-up activities. This interest and com- 
mitment provide the impetus for an immedi- 
ate start to the development phase. 


There are three stages to the development 
component: analysis and response, imple- 
mentation, and incorporation. The senior 
board official is responsible for providing 
leadership in each of these three stages, 
with assistance from the project co-ordinator 
and system personnel as needed. 


6. ANALYSIS AND RESPONSE 


The main goals of the analysis and response — 
stage are the dissemination of information 
to school system personnel and members of ~ 
the community, a critical review of report 
recommendations, and the development of _ 
procedures for the implementation of : 
changes in the system. 


As an outgrowth of the discussions following 
the presentation of the final evaluation 
report to the board, the senior board official, 
assisted by the project co-ordinator, 
develops alternative strategies for providing 
feedback to those who participated in the 
evaluation exercise and to other interested 
community groups. This feedback is planned — 
on a scale similar to the earlier system- d 
awareness activities with a view to maintain- © 
ing a high level of interest in development _ 
activities and to gaining support for them. 


The analysis and response stage links the 
evaluation phase and the development phase 
of the co-operative evaluation and develop- 
ment project. Therefore, it is an appropri- 
ate time for the board to publicize the 
evaluation findings and to declare its com- 
mitment to system renewal through the 
development activities. 


Under the leadership of the senior board 
official, an analysis and response committee 
is now established to assume the major 
responsibilities of this stage. The strategies 
suggested for the selection of the internal 
evaluation committee may be considered 
again for the establishment of the new com- 
mittee, which should broadly represent all 
members of the educational system. 


The responsibilities of the analysis and 
response committee are as follows: 


- to develop a classification system to deal 
with the issues raised by the evaluation 
report; 

- to provide clarification and interpretation 
of the final evaluation report; 

— to determine which changes have already 
taken place during the earlier evaluation 
stages; 

— to determine which issues still need 
attention; 

— to suggest alternative recommendations, 
if appropriate; 

— to suggest time lines, priorities, cost impli- 
cations, and the allocation of responsibility 
for follow-up action; and 

— to provide regular progress reports. 


The project co-ordinator and the internal 
evaluation chairperson are available, if 
required, to assist in the interpretation or 
clarification of particular aspects of the 
evaluation report. 


After the report of the analysis and response 
committee has been received and its recom- 
mendations reviewed, the senior board 
official submits a plan of action for the 
implementation stage to the board for its 
approval. 


A small, long-term, monitoring committee, 
representing the board, the administration, 
and the teaching staff and assisted by the 
project co-ordinator in a consultative role, 
is next established. The committee has the 
following duties: 


- to receive and consider progress reports 
during the implementation stage; 

— to report general progress to the board 
and other interested groups on a regular 
basis; and 

— to monitor on a long-term basis the devel- 
Opment activities. 


7. IMPLEMENTATION 


As its name implies, the goal of the imple- 
mentation stage is to carry out the recom- 
mended system changes according to the 
adopted plan of action. The responsibility 
for managing the implementation of the 
plan of action is assigned to regular line 
personnel in the system (e.g., supervisory 
officials, principals, curriculum committees, 
etc.) by the senior board official. Changes 
are implemented on a trial basis, where 
feasible, and progress reports are provided 
regularly to the monitoring committee. 


On completion of the implementation stage 


(this occurs at different times for different 
programs) the line personnel prepare formal 
reports that describe the results of the im-— 
plementation programs and provide rec- 
ommendations regarding possibilities for 


_ incorporation. These reports are submitted 


to the monitoring committee and the senior 


_ board official for review and presentation to 


the board. 


Successful implementation is a complex 
process. It involves the alteration of ma- 
terials, teaching approaches, and beliefs. 

A recent research report identified twelve 
factors affecting implementation.* The 
report includes a systematic summary of 
what implementation is, the factors that 
affect it, and guidelines for planning it. This 
report might prove useful for planners of the 
implementation stage of the CEDSS model. 


8. INCORPORATION 


The incorporation stage provides the board 
with the opportunity to review system 
changes and to effect decisions about con- 
tinuation, further modification, or cancel-— 
lation of the project. 


Many recommendations of the evaluation 
report are likely to be implemented and 
incorporated as permanent practice in one 
step. Major changes, however, may require 
extensive experimentation before becoming 
incorporated into the system. 


The monitoring committee, in consulta- 


tion with the senior board official, is 
responsible for reviewing implementation 


reports and preparing final recommendations 
to the board with regard to incorporation. 
The analysis and response committee also 
reviews the implemented programs to deter- 
mine whether the recommendations of the 
evaluation report have been carried out. 


Regardless of the final decision on the 
incorporation of any particular change, it 
may be well to review the benefits accruing 
to the system from the activities of the | 
implementation stage and the whole devel- 
opment component. The broad base of 
involvement of system personnel in imple- 
mentation activities has the potential for 
increasing their understanding of and level 
of commitment to the system as a whole. 
The development of new or improved 
teaching materials increases the bank of 
resources available to teachers and students. 
The in-service training and professional 
development activities lead to improved 
capability on the part of the participants. 


While each of these is important, probably 
the greatest long-term benefit of the whole 
process lies in the increased capacity of the 
school system to deal with all of the poten- 
tial challenges confronting it. 


4. M. Fullan and P. Park, Curriculum Implementation: 
A Resource Booklet (Toronto: Ministry of Education, 
Ontario, 1981). 
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| PPENDIX A: 


| THE FORMAT OF 
THE REPORTS 


A suggested format for each of the reports 
is outlined below. As indicated earlier, 
additional suggestions for writing a report 
are included in John Davis’s User’s Guide 
to the Program Review Surveys for Ele- 
mentary and Secondary Schools. 


THE INTERNAL 
EVALUATION REPORT 


The following is a suggested general format 
for the internal report: 


1. INTRODUCTION 

— names of the committee and subcom- 
mittee members 

— background information of a historical 
and socio-economic nature 

— calendar of activities of the internal 
evaluation committee, including samples 
of flyers, news releases, articles, etc. 

— strategies for data collection, including sam- 
ples of questionnaires and instruments used, 
the number of interviews conducted, etc. 


2. DESCRIPTION OF THE SCHOOL 
SYSTEM 


= organizational charts of the board and its 


staff 

— names, locations, and year of construc- 
tion of schools (including a map) 

— enrolments by grade or age 

— professional and non-professional staff 
lists, assignments, and timetables where 
appropriate 

~— budget summary 


3. EVALUATION FINDINGS 

— an overview of the system as perceived 
by (a) parents, (b) students, (c) teachers; 
a summary of the findings of the system- 
review survey 

— the evaluation of the roles of (a) trustees, 
(b) administrators, (c) principals, 
(d) teachers; a summary of the findings 
of the role-review survey 

— the general program evaluation for each 
division and for each of the evaluated 
subject areas; a summary of the findings 
of the program-review survey 

— the student-achievement assessment find- 
ings; a summary of the findings 


4. SUMMARY AND 
RECOMMENDATIONS 

The findings and the recommendations of 
the various subcommittees are summarized, 
and an attempt is made to place the recom- 
mendations in order of priority. 


Te: 
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THE EXTERNAL 
EVALUATION REPORT 


In goal-shared evaluation projects the exter- 


nal evaluation team has the responsibility 
for reviewing the report of the internal 
committee and for identifying and com- 
menting on important areas of concern that 
may have been omitted from the report of 
the internal evaluation committee. The ex- 
ternal team may find it helpful to complete 
its report as an exact parallel to the internal 
report, placing its comments right beside 
those of the internal team. 


The following is a suggested general format 
for the external report: : 


1. INTRODUCTION 

— names of the members of the external 
team 

— calendar of activities of the external team 

- information strategies for data collection, 
including the number of interviews, etc. 


2. EVALUATION FINDINGS 

This includes an overview of the system 
and should address areas on which the 
evaluation teams are in agreement or 
disagreement and areas that have been 
omitted. 


3. SUMMARY AND 
RECOMMENDATIONS 

The findings and recommendations are 
summarized. Some indication of priorities 
may be given. 


In goal-free evaluation reports the external 
evaluation team provides an unbiased ex- 
ternal assessment of the school system. It is 
suggested that the report follow the format 
above, but that in each of the areas evalu- 
ated the report identify areas of strength, 
areas of weakness, and recommendations 
for short- and long-range development 
activities. 


THE FINAL EVALUATION 
REPORT 


The project co-ordinator is responsible for 
the preparation of the final evaluation 
report, which is a compilation of the internal 
report, the external report, the minutes of - 
the co-operative review meeting, and the 
comments of the project co-ordinator. The 
following general format is suggested for the 
final report: 


— Part A: the observations, conclusions, 
and recommendations of the project co- 
ordinator; a summary of the recommenda- 
tions from the internal and external 
reports; 

- Part B: the internal report as submitted by 
the internal committee chairperson; 

— Part C: the external report as submitted 
by the external team chairperson; 

— Part D: the results of the co-operative 
review. 
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SCHOOL BOARD 
PERSONNEL 


ANALYSIS AND RESPONSE 
COMMITTEE 


ROLE 

This committee analyses and responds to 
the issues raised by the evaluation report 
and makes recommendations with regard to 
follow-up activities. 


FUNCTION 


The committee: 


— develops a classification system to deal 
with the issues raised by the evaluation 
report; 

— provides clarification and interpretation 


of the final evaluation report; 

| PPENDIX B: ROLE — determines which changes have already 
taken place; 

| DESCRIPTIONS — suggests alternative recommendations, 


if appropriate; 

— suggests time lines, priorities, cost implica- 
tions, and an allocation of responsibility 
for follow-up action; and 

— provides regular progress reports. 


SUGGESTED PROCEDURAL 
SCHEDULE 
It is suggested that the committee: 


1. analyse the findings and recommenda- 
tions of the evaluation report; 


2. determine a classification system to deal 
with the issues raised by the report; 


3. classify the issues; 


4. request clarification and interpretation, 
if required, from the project co-ordinator 
and the internal evaluation chairperson; 


5. provide progress reports identifying the 
changes that have already taken place; 


6. place the recommendations in a suggested 
order of priority; 


7. suggest a time line, cost implications 
(including materials, in-service training, 
etc.), and an allocation of responsibility 


for each recommendation; 
19 


8. prepare a report for the senior board 
official and the board that includes the 
information outlined in item 7 above for 
each of the recommendations of the eval- 
uation report and proposals for a plan of 
action for follow-up activities; 


9. after changes in programs have been 
implemented on a trial basis, review the 
reports of the monitoring committee with 
a view to determining whether the perti- 
nent recommendations of the evaluation 
report have been carried out. 


BOARD ADVISORY COMMITTEE 


This is an optional committee representing 
teaching personnel, support staff, adminis- 

trators, trustees, students, parents, and com- 
munity members. 


ROLE 

The board advisory committee serves as a 
consultative group to the senior board official 
regarding the various aspects of the CEDSS 
project, as requested, and serves as a liaison 
agent of the co-operative evaluation and 
development project with the groups repre- 
sented on the committee. 


FUNCTION 
The committee may provide suggestions 
regarding the following: 


~ plans for effecting awareness; 

— areas to be evaluated; 

— individuals who may be appointed to the 
internal committee and its subcommittees; 

— questionnaires and other data-collection 
strategies; . 

- individuals who may be appointed to 
development committees. 
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INITIATION TEAM 


This is an optional committee that includes 
members of the advisory committee or 
members selected by them, or individuals 
selected by the senior board official and the 
internal evaluation chairperson. 


ROLE 

The initiation team assists the internal 
evaluation chairperson in effecting system 
and community awareness and in determin- 
ing areas to be evaluated. 


FUNCTION 


The committee: 


— prepares materials and strategies for 
effecting system and community awareness 
and determining areas to be evaluated; 

— visits schools, addresses parent groups and 
community organizations, and so on; | 

— arranges for input regarding areas to be 
evaluated from school personnel, support 
staff, students, parents, and community 
members; 

— arranges for communication, to personnel 
throughout the school system and to 
parents and community members, of the 
final list of areas to be evaluated. 


INTERNAL EVALUATION 
COMMITTEE AND SUBCOMMITTEES 


FUNCTION 


The committee and its subcommittees: 


- plan the internal evaluation stage; 

— provide leadership in the implementation 
of the internal evaluation; 

— keep all levels of staff informed, on a 
regular basis, regarding the plans and 
progress of the internal evaluation com- 
mittee; 

— prepare the internal evaluation report for 
the project co-ordinator; 

— participate in the co-operative review 
(internal evaluation committee members 
only). 


ROLE OF THE INTERNAL 
EVALUATION CHAIRPERSON 


- The chairperson makes plans for and pro- 


vides leadership in effecting system and 


- community awareness of the co-operative 
~ evaluation project; makes plans for and 


provides leadership in determining areas to 


_ be evaluated; and assumes responsibility 
_ for the internal evaluation of the school 
_ system. 


f SUGGESTED PROCEDURAL 


SCHEDULE - INTERNAL 


~ EVALUATION CHAIRPERSON 
It is suggested that the chairperson: 


1. submit written acceptance of his/her 
appointment to the senior board official; 


2. with the assistance of the senior board 
official and the project co-ordinator, 
plan and implement strategies for effect- 
ing system and community awareness 
and for determining areas to be 
evaluated; 


3. in consultation with the senior board 
official and the project co-ordinator: 


a) determine the design of the internal 
evaluation (areas of emphasis); 

b) select internal evaluation committee 
and subcommittee members (the 
number will vary with the size of 
the jurisdiction); 

c) plan and implement an orientation 
program for the internal evalua- 
tion committee and subcommittee 
members; 


4. provide leadership in the planning of 
internal evaluation activities and direct 
the work of the internal evaluation 
committee and subcommittees; 


5. in consultation with the project co- 
ordinator, involve the internal evalua- 
tion committee in finalizing the design 
of the internal evaluation, organizing « 
the subcommittees, establishing time 
lines, selecting evaluation instruments, 
projecting cost allocations, and so on; 


6. finalize internal evaluation plans with 
the senior board official; 


7. arrange for regular communication with 
all segments of the community; 


8. co-ordinate the activities of the internal 
evaluation; 


9. prepare the internal evaluation 
report and forward it to the project 
co-ordinator; 


10. serve as chief spokesperson for the 
internal evaluation committee during 
the co-operative review; 


11. assist the project co-ordinator in the 
presentation of the final report to the 
board, as required; 


12. participate, as requested, in workshops 
designed to provide the ministry with 
feedback on the effectiveness of the 
evaluation models being employed; 


13. participate in a consultative capacity 
in the activities of the analysis and 
response stage of the development 
component of the CEDSS model. 


SUGGESTED PROCEDURAL 
SCHEDULE - INTERNAL EVALUATION 
COMMITTEE MEMBER 


It is suggested that a committee member: 


1. submit written acceptance of his/her ap- 
pointment to the committee chairperson; 


2. participate in an orientation program; 


3. assist in planning and implementing 
the internal evaluation (e.g., planning 
activities and strategies, selecting or 
developing evaluation instruments); 
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4. participate in the planning and imple- 
mentation of an ongoing communication 
program for trustees, administrators, staff, 
and the community; 


5. participate in the collection, analysis, 
and interpretation of data as required; 


6. participate in the preparation of the first 
draft of the internal evaluation report; 


7. review and comment on the draft of the 
report; 


8. participate in meetings to discuss and 
finalize the internal report; 


9. participate in the co-operative review. 
One internal evaluation committee 
member is assigned the task of keeping 
minutes of the co-operative review meet- 
ing and submitting a copy of the minutes 
to the project co-ordinator. 


SUGGESTED PROCEDURAL 
SCHEDULE - INTERNAL EVALUATION 
SUBCOMMITTEE MEMBER 


It is suggested that a subcommittee member: 


1. submit written acceptance of his/her ap- 
pointment to the committee chairperson; 


2. participate in an orientation program; 


in consultation with the internal 
evaluation committee, plan evaluation 
activities in the assigned area of evalua- 
tion (e.g., selecting or developing 
instruments, determining population 

to be sampled); 


4. in consultation with the internal evalua- 
tion committee, implement evaluation 
activities; 
analyse and interpret evaluation data; 


6. prepare the subcommittee report for the 
internal evaluation committee chair- 
person. 


MONITORING COMMITTEE 


ROLE | 

This committee monitors the activities of 
the implementation stage and provides 
recommendations to the board regarding 
the incorporation or the cancellation of 
new or adapted programs. 


FUNCTION 

The committee: 

~ receives and considers progress reports 
during the implementation stage; 

— reports general progress to the board and 
other interested groups on a regular basis; 

—- monitors on a long-term basis the develop- 
ment activities; 

— makes recommendations regarding the 
incorporation, cancellation, or further 
modification of new or adapted programs. 


SUGGESTED PROCEDURAL 
SCHEDULE | 

It is suggested that the monitoring com- 
mittee: 


1. choose a chairperson for the committee; 


2. become familiar with all aspects of the 
board’s plan of action for the implemen- 
tation stage; 


3. review the factors that research for the 
project has identified as affecting imple- 
mentation; 


4. establish criteria for the evaluation of: 
implementation activities; 


Cn 


review progress reports as they are 
received from line personnel; 


6. identify, from progress reports, activities 
that do not meet the evaluation criteria; 


7. develop and implement a regular pat- 
tern for the reporting of general progress 
to the board and other interested groups; 


8. review the formal reports on implemented 
programs in terms of the established 
evaluation criteria and make recom- 
mendations to the board and the senior 
board official regarding the incorporation, 
modification, or cancellation of the new 
programs. 


_ RESEARCH DIRECTOR 


SUGGESTED PROCEDURAL 
SCHEDULE 
ROLE It is suggested that the research director: 
_ The research director provides research ex- 1 x ; 
. assist the internal evaluation chair- 


_ pertise that will ensure that the information 
_ provided by the co-operative evaluation is 


both valid and reliable. 


Note: Some large urban boards may have a 
_ research director and research staff. A 
_ possible contribution that such a research 


person in carrying out a procedure for 
determining areas to be evaluated; 


2. assist the internal evaluation chair- 
person in the detailed design of the 
internal evaluation; 


_ department can make to the CEDSS project 3. determine appropriate types of instru- 
_ is outlined below. mentation to be used in the internal 
evaluation and direct any necessar 

- FUNCTION i 


modification of the instruments pro- 


The research director: 


~- works in liaison with the senior board 
official; 

- provides direction and supervision to any 
research staff that may be assigned to the 
CEDSS project; 
= participates in: 


a) the determination of areas to be 

evaluated; 

b) the development and/or modification 
of assessment instruments; 

c) the analysis and interpretation of data; 

d) the writing of the internal report; 

e) the establishment of the design of the 
internal and external evaluations; 

f) the provision of input throughout the 
internal and external evaluations, as 
requested; 

g) the co-operative review; 

h) the follow-up on development 
activities. 


vided through the ministry or the devel- 
opment of new instruments; 


4. provide consultative services to the 
internal evaluation chairperson and in- 
ternal evaluation committees regarding 
data analysis and interpretation, the 
writing of the internal report, and other 
services, as required; } . 


5. assist in the design of the external 
evaluation; 


6. be available for consultation with the 
external evaluation team, as required; 


7. participate in and provide consultative 
services during the co-operative review; 


8. assist the project co-ordinator and senior 
board official in the presentation of the 
final report, as required; 


9. serve on the analysis and response com- 
mittee, which undertakes an in-depth 
discussion of the final report; 


10. provide consultative services, as required, 
during the implementation activities; 


11. participate, as requested, in workshops 
designed to provide the ministry with 
feedback on the effectiveness of the 
evaluation models being employed. 
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SENIOR BOARD OFFICIAL 


ROLE . 

The senior board official provides local 
leadership throughout the CEDSS project 
and co-ordinates each of the three stages of 
the development component of the CEDSS 
model. 


FUNCTION 


The senior board official: 


~— meets with local officials and staff of the 
board, as well as with ministry officials; 

— makes progress reports to the board; 

— participates in: 


a) the selection of the project 
co-ordinator and, where warranted, 
an assistant project co-ordinator; 

b) the development of materials and 
strategies for effecting system and com- 
munity awareness; 

c) the determination of areas to be 
evaluated; 

d) the selection of the internal evalua- 
tion committee chairperson and the 
committee members; 

e) the orientation of internal evalua- 
tion committee and subcommittee 
members; 

f) the establishment of the design of the 
internal and external evaluations; 

g) the selection of the external evalua- 
tion chairperson and team members; 

h) the internal and external evaluation 
phases, as requested. 


— reviews the final evaluation report with 
the project co-ordinator; 

— participates in the presentation to the 
board of the final evaluation report; 

— plans and provides leadership for each of 
the three stages of the development 
component; 

— plans the distribution of the final report. 


—~ 


SUGGESTED PROCEDURAL 
SCHEDULE 
It is suggested that the senior board official: 


1. arrange and chair a meeting of local 
officials and staff of the board and 
ministry officials to discuss the param- 
eters of the CEDSS project; 


2. a) participate in the preparation and 
exchange of letters of agreement 


between the board and the ministry 


after the board has passed a resolu- 
tion indicating its commitment to a 
co-operative evaluation and develop- 
ment project; 

b) participate in discussion with the 
ministry regarding the selection of 
the project co-ordinator and, where 
required, an assistant project 
co-ordinator; 


3. assist the internal evaluation chair- 
person and initiation team in effecting 
system and community awareness and 
in determining areas to be evaluated; 


4. plan and carry out a procedure for the 
selection of the internal evaluation 
chairperson; 


5. assist the internal evaluation chair- 
person in the selection of the internal 


evaluation committee and subcommittee © 


“members; 


6. assist the internal evaluation chair- 
person and project co-ordinator in the 
orientation and in-service training of 
internal committee and subcommittee 
members; 


7. assist the internal evaluation chair- 
person and the project co-ordinator in 
the establishment of the sigs design 
of the evaluation; 


8. provide regular progress reports to the 
chief education officer and the board; 


. provide a section in the internal report 
in which the achievements of the last 
five years are discussed from his/her 
perspective; 


. assist-the project co-ordinator in the 
selection of the external evaluation 
chairperson and team members; 


. assist in the design of the external 
evaluation; 


- provide input during the external eval- 
uation, as requested; ok 


. participate in the co-operative review; 


. discuss the final evaluation Teport with 
the project co-ordinator; 


. discuss the final evaluation report with 
the chief education officer and distrib- 
ute copies of the report to the members 


of the board; 


. assist the project co-ordinator in the 
presentation of the final evaluation | 
report to the board; 


. complete the “Grant for a Co-operative 


Evaluation and Development of School 
Systems Project” forms and submit 
them to the regional director of educa- 
tion when appropriate. 


with assistance from the project co- 
ordinator and the internal committee 
chairperson, plan the release and distri- 
bution of the final report and plan and 
carry out strategies to provide feedback 
to all participants in the evaluations 
and to other interested community 
groups; 


. establish an analysis and response 
committee; 


. review the report and recommended 
plan of action of the analysis and 
response committee; 


adopt a plan of action for the imple- 
mentation stage in consultation with 
other senior officials of the board; 


. establish a small, foe TEETH monitoring 


committee; 


assign the management of the imple- 
mentation of the plan of action to 


| regular line personnel in the system; 


é) provide regular f progress reports to the 


board; 
initiate activities designed to facilitate 
implementation of the plan of action; 


review, in co-operation with the mon- 


itoring committee, the reports of the 


implementation programs and present 
the results to the board; 


Ue ocsule widwiene monitoring committee 


on recommendations to the board with 


regard to the incorporation of new or 
adapted programs; 


determine, with the help of the analy- 
sis and response committee, whether 


the implemented programs have fulfilled 


the recommendations of the evaluation 
report; 


. maintain regular liaison with ministry 


officials regarding progress being made 
in the development activities; 


. participate, as requested, in workshops 


designed to provide the ministry with 
feedback on the effectiveness of the 
CEDSS model and related strategies. 
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MINISTRY AND OTHER 
PERSONNEL 


PROJECT CO-ORDINATOR 


ROLE 

The project co-ordinator assumes respon- 
sibility for the CEDSS project on behalf of 
the ministry. 


FUNCTION 


The project co-ordinator: 


— meets with the board, its officials, and 
the regional director; 

— assists in the establishment of the design 
of the internal evaluation; 

~ builds the external evaluation team; 

— receives the internal and external reports; 

— plans and leads the co-operative review; 

— prepares the final evaluation report; 

- presents the final evaluation report to the 
senior board official and the board; 

— assists in a consultative capacity during 
the three stages of the development 
component. 
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SUGGESTED PROCEDURAL 
SCHEDULE, + 


It is suggested that the project co-ordinator: 


I 


meet with the board and its officials to 
clarify the parameters of the CEDSS 
project, the nature of both board and 
ministerial involvement, and the type 
of report and follow-up service to be 
expected; 


. arrange for the exchange of letters of 


agreement between the ministry and 
the board after the board has passed a 
resolution indicating its commitment 
to a co-operative evaluation and devel- 


opment project; 


. maintain liaison with the regional direc- 


tor regarding the project; 


assist local personnel in effecting system 
and community awareness; 


assist the internal evaluation chair- 
person in the design of the internal 
evaluation and in the carrying out of 
evaluation plans; 


. provide in-service training for the in- 


ternal evaluation committee and sub- 
committee members; 


provide the internal evaluation commit- 
tee with evaluation instruments availa- 
ble from the ministry (e.g., role-review 
surveys, program-review surveys, system- 
review surveys, questionnaires from 
provincial reviews, instruments of the 
Ontario Assessment Instrument Pool); 


. assist the internal evaluation com- 


mittee in the development of locally 
produced questionnaires and other 
evaluation instruments; 


assist the senior board official in the 
preparation of the application for the 
“Grant for a Co-operative Evaluation 
and Development of School Systems 
Project”; 


7 
| 
‘. 
; 
, 
y 


a 


14. 


10. 


py 


Loe 


establish, in consultation with the in- 
ternal evaluation chairperson and the 
senior board official, the design of the 
external evaluation; 


select the members of the external 
evaluation team in consultation with 
local personnel. The number of team 
members and the duration of their visit 
varies with the size of the jurisdiction 
involved. The project co-ordinator 


-should ensure that appropriate letters 


of agreement and a financial under- 
standing with respect to members of 
the external team are formalized. 
(See Appendix C for procedures for 
establishing the external team.); 


make progress reports as required; 


. receive the report of the internal evalu- 


ation committee and forward copies to 
the external evaluation team; 


arrange a meeting involving the internal 
evaluation committee and the external 
team chairperson to clarify the areas to 
be evaluated; 


plan and lead an orientation workshop 

for members of the external evaluation 

team. This workshop might include the 
following: 


- clarification of the objectives and 
process of CEDSS; 

- general information on the jurisdiction; 

— study and discussion of the internal 
report; 

- strategy for the visit (emphasis, 
timetables, etc.); 

- distribution of maps of the jurisdiction; 

- distribution of a list of available 
accommodation; | 
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be available for consultation with the 
external evaluation team as required; 


receive the report of the external 
evaluation team and forward copies to 
the internal evaluation chairperson; 


arrange and lead the co-operative review 
meetings. If necessary, further visitation, 
reports, and meetings may be scheduled 

to resolve areas of disagreement; 


arrange for the provision of meeting 
rooms, secretarial services, and expense 
claims for the external team; 


prepare the final evaluation report, in- 
cluding a summary of the major issues, 
and forward official copies to both the 
board and the ministry; 


present the final evaluation report to 
the board in co-operation with the 
senior board official; 


be available for consultation with the 
board during the activities of the devel- 
opment component; 


assist the senior board official to develop 
strategies for distributing the report to 
the public and to provide feedback to 
the participants in the evaluation 
exercise; 


be available for consultation with the 
monitoring committee during the imple- 
mentation stage of the development 
component; 


participate, as requested, in workshops 
designed to provide the ministry with 
feedback on the effectiveness of the 
evaluation model and the strategies 
employed. 
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ASSISTANT PROJECT 


CO-ORDINATOR 


ROLE . 
The assistant project co-ordinator assists 
the project co-ordinator in leading and as-_ 


suming responsibility for the CEDSS protest a 


on behalf of the ministry. 


FUNCTION | 

The assistant project co- MS will be 
aware of all of the activities of the CEDSS 
project and will assist the project co- 
ordinator and internal evaluation chair-_ 
person as Laney 


SUGGESTED PROCEDURAL 


 SOREDULE 


28 


It is suggested that the assistant project 
co-ordinator: ; 


1. become families with all i the activities 
and responsibilities of the project co- 
ordinator, as described in this manual; 


2. assist the project co-ordinator in the i 
performance of all aspects of his/her 
responsibilities in the CEDSS project; 


3. carry out such specific tasks and respon- 


sibilities as may be assigned from time 
to time by the project co-ordinator. 


. FUNCTION - 


Lhe external evaluation team: 


~ reviews ‘the report of the internal ee 
tion committee (in goal-shared projects 
only); | | 


= visits schools; | 
fo - confers with local Derspuael 


— prepares the external evaluation report 
based on its observations and conclusions 
and submits the report to the i ies 
‘co-ordinator; — 


= ~ participates in the co- Spetene review 


meeting. 


t ROLE OF THE EXTERNAL 
| EVALUATION CHAIRPERSON 


_ The external evaluation chairperson leads 


and assumes responsibility for an external 


evaluation of a school system. 


“SUGGESTED PROCEDURAL 1 

| SCHEDULE - EXTERNAL EVALUATION | 
CHAIRPERSON 

| It is suggested that the external evaluation 

| ne 


. submit written acceptance of his/her 
appointment to the project co-ordinator; 


2. assist the project co-ordinator in deter- 
mining the design of the external 
evaluation and in building the external 
evaluation team; 


3. assist the project co-ordinator in plan- 


ning and implementing the orientation 
program for the members of the external 
evaluation team; 


4. serve as leader of the external team q 
during its visit to the school system, and — 
chair meetings to receive reports, discuss — 
progress, make any necessary adjust- 
ments in plans, and evaluate and report 
on aspects of the system being reviewed; 


5. prepare a draft of the external evalua- 


tion report and distribute copies to all 
team members; 


- 1 


t 6. 


chair a meeting of the Seis evalua- 
tion team to discuss, draft, and finalize 
the report; 


submit the report of the external evalu- 
ation team to the project co-ordinator; 


in consultation with the project co- 
ordinator, plan the role of the external 
team for the co-operative review; 


serve as chief spokesperson for the 
external team during the co-operative 


review meeting; 


. at the request of the co-ordinator, | 


arrange for supplementary visits that | 
may be required to clarify areas of dis- 
agreement; 


assist the project co- eta oe in the 
presentation of the final evaluation 
report to the board as required; 


. participate, as requested, in workshops 


designed to provide the ministry with 


feedback on the effectiveness of the 


evaluation models employed. 


SUGGESTED PROCEDURAL 
SCHEDULE ~ EXTERNAL EVALUATION 
TEAM MEMBER 


It is suggested that the committee member: 


a) 


submit written acceptance of his/her ap- 
pointment to the project co-ordinator; 


participate in an orientation program; 


3. study carefully the report of the internal 


evaluation committee before the formal 
visitation (in goal-shared projects); 


. participate in formal visits and related 
activities. These will include an organi- 


zational meeting, visits to schools, 
classrooms, board offices, and so on; 
and reporting to the external team 
chairperson regarding observations and 
conclusions as required; 


participate in the preparation . the first 
draft of the external evaluation report; 


review and comment on the draft of the 
report; 


. participate in meeting(s) to discuss and 


finalize the external report; 


. on the request of the project co-ordinator 


or team chairperson, participate in 

the co-operative review meeting. One 
external evaluation team member is 
assigned the task of keeping minutes of 


the co-operative review meeting and 


of submitting a copy of the minutes 
to the project co-ordinator; 


. at the request of the project co-ordinator 


or external team chairperson, make addi- 
tional visits that may be required to 
clarify areas of disagreement. 
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| PPENDIX C: 


PROCEDURES FOR 
BUILDING THE 
EXTERNAL 
EVALUATION 
TEAM 


The project co-ordinator is responsible for 
the selection of the chairperson and mem- 
bers of the external evaluation team after 
consultation with the chairperson of the 
internal committee and the senior board 
official. 


The following are the procedures for build- 
ing the external team: 


1. Determine the number of evaluator days 


to be allocated, according to the follow- 

ing formula: 

Number of evaluator days = 1 evaluator day per 
200 students en- 
rolled within the 
board (with a mini- 
mum of 50 anda 
maximum of 200 
evaluator days) 


. Determine the size of the external evalu- 


ation team (a minimum of ten and a 
maximum of twenty evaluators) and the 
duration of the visit (a minimum of five 
and a maximum of ten days). Note: The 
size of the team multiplied by the dura- 
tion of the visit equals the number of 
evaluator days. 


. In consultation with the chairperson of 


the internal evaluation committee and 
the senior board official: 


a) determine the range of qualifications 
and experience required; 


b) build a list of acceptable evaluators. 
Evaluators may be chosen from min- 
istry personnel, board personnel, 
OISE personnel, personnel in the | 
faculties of education, practising 
teachers, and members of the public. 


. Arrange through directors, regional 


directors, and other officials for the 
appointment of appropriate evaluators 
(as identified in item 3 above) to the 
external team. 
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APPENDIX D: 


) PROPOSED TIME 
| LINE FOR THE 

| EVALUATION 
COMPONENT 


When a specific time line is 
planned, consideration should be 
given to holiday periods, availability 
of key personnel, semester breaks, 
traditional examination periods, 
and so on. 
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Mar. — Apr. Board makes a commitment 
to the CEDSS project. 


May — June _ Leadership positions are 
assigned, and materials and strategies for 
effecting system and community awareness 
and for determining areas to be evaluated 
are prepared. 


Sept. —- Nov. System and community 
awareness is developed, and the areas to 
be evaluated are determined. 


Nov. Internal evaluation committee and 
subcommittee members are selected. An 
orientation program for these members is 
implemented. 


Nov. = Dec. The evaluation instruments 
to be used are determined. Attitude 
surveys and role-review surveys are 
administered. 


Jan. — Feb. Data relating to all areas to be 
evaluated are collected and analysed. 
Subcommittee reports are prepared. 


Late Feb. - early Mar. The internal evalua- 
tion report is prepared and submitted to 
the project co-ordinator. 


Apr. The external evaluation is conducted. 
The co-operative review meeting is held. 


May The final evaluation report is prepared 
and presented to the board. 


June The analysis of the final report 
begins. Feedback is sent out to all partici- 
pants in the evaluation, and follow-up 
activities are discussed. 


Sept. The analysis of the final report con- 
tinues. The activities of the development 
component begin. 
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